The significance of models of vision for the development of artificial handwriting recognition systems

Artificial Handwriting Recognition (AHR) systems are currently developed in a largely ad hoc fashion. The central premise of this work is the need to return to first principles and identify an underlying rationale for the representations used in handwriting recognition. An interdisciplinary approach is advocated that combines the perspectives of cognitive science and pattern recognition engineering. Existing surveys of handwriting recognition are reviewed and an information-space analogy is presented to model how features encode evidence. Handwriting recognition is treated as an example of a simple visual task that uses a limited set of our visual abilities based on the observations that i) biological systems provide an example of a successful handwriting recognition system, and ii) vision is a prerequisite of recognition. A set of six feature types for which there is empirical evidence of their detection in early visual processing is identified and a layered framework for handwriting recognition is proposed that unifies the perspectives of cognitive science and engineering. The outer layers of the framework relate to the capture of raw sensory data and feature extraction. The inner layers concern the derivation and comparison of structural descriptions of handwriting. The implementation of an online AHR system developed in the context of the framework is reported. The implementation uses a fuzzy graph-based approach is used to represent structural descriptions of characters. Simple directed graphs for characters are compared by searching for subgraph isomorphisms between input characters and know prototypes. Trials are reported for a test set of 1000 digits drawn from 100 different subjects using a KNearest Neighbour approach (KNN) approach to classification. For K=3, the mean recognition accuracy is 68.3% and for K=5 it is 70.7%. Linear features were found to be the most significant. The work concludes that the current understanding of visual cognition is incomplete but does provide a basis for the development of artificial handwriting recognition systems although their performance is currently less than that of existing engineered systems.